An acoustic environment is often noisy, making it difficult to reliably detect and react to a desired informational signal. In one particular example, a speech signal is generated in a noisy environment, and speech processing methods are used to separate the speech signal from the environmental noise. Such speech signal processing is important in many areas of everyday communication, since noise is almost always present in real-world conditions. Noise is defined as the combination of all signals interfering or degrading the speech signal of interest. The real world abounds from multiple noise sources, including single point noise sources, which often transgress into multiple sounds resulting in reverberation. Unless separated and isolated from background noise, it is difficult to make reliable and efficient use of the desired speech signal. Background noise may include numerous noise signals generated by the general environment, signals generated by background conversations of other people, as well as reflections and reverberation generated from each of the signals. In communication where users often talk in noisy environments, it is desirable to separate the user's speech signals from background noise. Speech communication mediums, such as cell phones, speakerphones, headsets, cordless telephones, teleconferences, CB radios, walkie-talkies, computer telephony applications, computer and automobile voice command applications and other hands-free applications, intercoms, microphone systems and so forth, can take advantage of speech signal processing to separate the desired speech signals from background noise.
Many methods have been created to separate desired sound signals from background noise signals, including simple filtering processes. Prior art noise filters identify signals with predetermined characteristics as white noise signals, and subtract such signals from the input signals. These methods, while simple and fast enough for real time processing of sound signals, are not easily adaptable to different sound environments, and can result in substantial degradation of the speech signal sought to be resolved. The predetermined assumptions of noise characteristics can be over-inclusive or under-inclusive. As a result, portions of a person's speech may be considered “noise” by these methods and therefore removed from the output speech signals, while portions of background noise such as music or conversation may be considered non-noise by these methods and therefore included in the output speech signals.
In signal processing applications, typically one or more input signals are acquired using a transducer sensor, such as a microphone. The signals provided by the sensors are mixtures of many sources. Generally, the signal sources as well as their mixture characteristics are unknown. Without knowledge of the signal sources other than the general statistical assumption of source independence, this signal processing problem is known in the art as the “blind source separation (BSS) problem”. The blind separation problem is encountered in many familiar forms. For instance, it is well known that a human can focus attention on a single source of sound even in an environment that contains many such sources, a phenomenon commonly referred to as the “cocktail-party effect.” Each of the source signals is delayed and attenuated in some time varying manner during transmission from source to microphone, where it is then mixed with other independently delayed and attenuated source signals, including multipath versions of itself (reverberation), which are delayed versions arriving from different directions. A person receiving all these acoustic signals may be able to listen to a particular set of sound source while filtering out or ignoring other interfering sources, including multi-path signals.
Considerable effort has been devoted in the prior art to solve the cocktail-party effect, both in physical devices and in computational simulations of such devices. Various noise mitigation techniques are currently employed, ranging from simple elimination of a signal prior to analysis to schemes for adaptive estimation of the noise spectrum that depend on a correct discrimination between speech and non-speech signals. A description of these techniques is generally characterized in U.S. Pat. No. 6,002,776 (herein incorporated by reference). In particular, U.S. Pat. No. 6,002,776 describes a scheme to separate source signals where two or more microphones are mounted in an environment that contains an equal or lesser number of distinct sound sources. Using direction-of-arrival information, a first module attempts to extract the original source signals while any residual crosstalk between the channels is removed by a second module. Such an arrangement may be effective in separating spatially localized point sources with clearly defined direction-of-arrival but fails to separate out a speech signal in a real-world spatially distributed noise environment for which no particular direction-of-arrival can be determined.
Methods, such as Independent Component Analysis (“ICA”), provide relatively accurate and flexible means for the separation of speech signals from noise sources. ICA is a technique for separating mixed source signals (components) which are presumably independent from each other. In its simplified form, independent component analysis operates an “un-mixing” matrix of weights on the mixed signals, for example multiplying the matrix with the mixed signals, to produce separated signals. The weights are assigned initial values, and then adjusted to maximize joint entropy of the signals in order to minimize information redundancy. This weight-adjusting and entropy-increasing process is repeated until the information redundancy of the signals is reduced to a minimum. Because this technique does not require information on the source of each signal, it is known as a “blind source separation” method. Blind separation problems refer to the idea of separating mixed signals that come from multiple independent sources.
Many popular ICA algorithms have been developed to optimize their performance, including a number which have evolved by significant modifications of those which only existed a decade ago. For example, the work described in A. J. Bell and T J Sejnowski, Neural Computation 7:1129-1159 (1995), and Bell, A. J. U.S. Pat. No. 5,706,402, is usually not used in its patented form. Instead, in order to optimize its performance, this algorithm has gone through several recharacterizations by a number of different entities. One such change includes the use of the “natural gradient”, described in Amari, Cichocki, Yang (1996). Other popular ICA algorithms include methods that compute higher-order statistics such as cumulants (Cardoso, 1992; Comon, 1994; Hyvaerinen and Oja, 1997).
However, many known ICA algorithms are not able to effectively separate signals that have been recorded in a real environment which inherently include acoustic echoes, such as those due to room architecture related reflections. It is emphasized that the methods mentioned so far are restricted to the separation of signals resulting from a linear stationary mixture of source signals. The phenomenon resulting from the summing of direct path signals and their echoic counterparts is termed reverberation and poses a major issue in artificial speech enhancement and recognition systems. ICA algorithms may require long filters which can separate those time-delayed and echoed signals, thus precluding effective real time use.
Known ICA signal separation systems typically use a network of filters, acting as a neural network, to resolve individual signals from any number of mixed signals input into the filter network. That is, the ICA network is used to separate a set of sound signals into a more ordered set of signals, where each signal represents a particular sound source. For example, if an ICA network receives a sound signal comprising piano music and a person speaking, a two port ICA network will separate the sound into two signals: one signal having mostly piano music, and another signal having mostly speech.
Another prior technique is to separate sound based on auditory scene analysis. In this analysis, vigorous use is made of assumptions regarding the nature of the sources present. It is assumed that a sound can be decomposed into small elements such as tones and bursts, which in turn can be grouped according to attributes such as harmonicity and continuity in time. Auditory scene analysis can be performed using information from a single microphone or from several microphones. The field of auditory scene analysis has gained more attention due to the availability of computational machine learning approaches leading to computational auditory scene analysis or CASA. Although interesting scientifically since it involves the understanding of the human auditory processing, the model assumptions and the computational techniques are still in its infancy to solve a realistic cocktail party scenario.
Other techniques for separating sounds operate by exploiting the spatial separation of their sources. Devices based on this principle vary in complexity. The simplest such devices are microphones that have highly selective, but fixed patterns of sensitivity. A directional microphone, for example, is designed to have maximum sensitivity to sounds emanating from a particular direction, and can therefore be used to enhance one audio source relative to others. Similarly, a close-talking microphone mounted near a speaker's mouth may reject some distant sources. Microphone-array processing techniques are then used to separate sources by exploiting perceived spatial separation. These techniques are not practical because sufficient suppression of a competing sound source cannot be achieved due to their assumption that at least one microphone contains only the desired signal, which is not practical in an acoustic environment.
A widely known technique for linear microphone-array processing is often referred to as “beamforming”. In this method the time difference between signals due to spatial difference of microphones is used to enhance the signal. More particularly, it is likely that one of the microphones will “look” more directly at the speech source, whereas the other microphone may generate a signal that is relatively attenuated. Although some attenuation can be achieved, the beamformer cannot provide relative attenuation of frequency components whose wavelengths are larger than the array. These techniques are methods for spatial filtering to steer a beam towards a sound source and therefore putting a null at the other directions. Beamforming techniques make no assumption on the sound source but assume that the geometry between source and sensors or the sound signal itself is known for the purpose of dereverberating the signal or localizing the sound source.
Another known technique is a class of active-cancellation algorithms, which is related to sound separation. However, this technique requires a “reference signal,” i.e., a signal derived from only of one of the sources. Active noise-cancellation and echo cancellation techniques make extensive use of this technique and the noise reduction is relative to the contribution of noise to a mixture by filtering a known signal that contains only the noise, and subtracting it from the mixture. This method assumes that one of the measured signals consists of one and only one source, an assumption which is not realistic in many real life settings.
Techniques for active cancellation that do not require a reference signal are called “blind” and are of primary interest in this application. They are now classified, based on the degree of realism of the underlying assumptions regarding the acoustic processes by which the unwanted signals reach the microphones. One class of blind active-cancellation techniques may be called “gain-based” or also known as “instantaneous mixing”: it is presumed that the waveform produced by each source is received by the microphones simultaneously, but with varying relative gains. (Directional microphones are most often used to produce the required differences in gain.) Thus, a gain-based system attempts to cancel copies of an undesired source in different microphone signals by applying relative gains to the microphone signals and subtracting, but not applying time delays or other filtering. Numerous gain-based methods for blind active cancellation have been proposed; see Herault and Jutten (1986), Tong et al. (1991), and Molgedey and Schuster (1994). The gain-based or instantaneous mixing assumption is violated when microphones are separated in space as in most acoustic applications. A simple extension of this method is to include a time delay factor but without any other filtering, which will work under anechoic conditions. However, this simple model of acoustic propagation from the sources to the microphones is of limited use when echoes and reverberation are present. The most realistic active-cancellation techniques currently known are “convolutive”: the effect of acoustic propagation from each source to each microphone is modeled as a convolutive filter. These techniques are more realistic than gain-based and delay-based techniques because they explicitly accommodate the effects of inter-microphone separation, echoes and reverberation. They are also more general since, in principle, gains and delays are special cases of convolutive filtering.
Convolutive blind cancellation techniques have been described by many researchers including Jutten et al. (1992), by Van Compernolle and Van Gerven (1992), by Platt and Faggin (1992), Bell and Sejnowski (1995), Torkkola (1996), Lee (1998) and by Parra et al. (2000). The mathematical model predominantly used in the case of multiple channel observations through an array of microphones, the multiple source models can be formulated as follows:
            x      i        ⁡          (      t      )        =                    ∑                  i          =          0                L            ⁢                        ∑                      j            =            1                    m                ⁢                                            a              ijl                        ⁡                          (              t              )                                ⁢                                    s              j                        ⁡                          (                              t                -                l                            )                                            +                  n        i            ⁡              (        t        )            
where the x(t) denotes the observed data, s(t) is the hidden source signal, n(t) is the additive sensory noise signal and a(t) is the mixing filter. The parameter m is the number of sources, L is the convolution order and depends on the environment acoustics and t indicates the time index. The first summation is due to filtering of the sources in the environment and the second summation is due to the mixing of the different sources. Most of the work on ICA has been centered on algorithms for instantaneous mixing scenarios in which the first summation is removed and the task is to simplified to inverting a mixing matrix a. A slight modification is when assuming no reverberation, signals originating from point sources can be viewed as identical when recorded at different microphone locations except for an amplitude factor and a delay. The problem as described in the above equation is known as the multichannel blind deconvolution problem. Representative work in adaptive signal processing includes Yellin and Weinstein (1996) where higher order statistical information is used to approximate the mutual information among sensory input signals. Extensions of ICA and BSS work to convolutive mixtures include Lambert (1996), Torkkola (1997), Lee et al. (1997) and Parra et al. (2000).
ICA and BSS based algorithms for solving the multichannel blind deconvolution problem have become increasing popular due to their potential to solve the separation of acoustically mixed sources. However, there are still strong assumptions made in those algorithms that limit their applicability to realistic scenarios. One of the most incompatible assumption is the requirement of having at least as many sensors as sources to be separated. Mathematically, this assumption makes sense. However, practically speaking, the number of sources is typically changing dynamically and the sensor number needs to be fixed. In addition, having a large number of sensors is not practical in many applications. In most algorithms a statistical source signal model is adapted to ensure proper density estimation and therefore separation of a wide variety of source signals. This requirement is computationally burdensome since the adaptation of the source model needs to be done online in addition to the adaptation of the filters. Assuming statistical independence among sources is a fairly realistic assumption but the computation of mutual information is intensive and difficult. Good approximations are required for practical systems. Furthermore, no sensor noise is usually taken into account which is a valid assumption when high end microphones are used. However, simple microphones exhibit sensor noise that has to be taken care of in order for the algorithms to achieve reasonable performance. Finally most ICA formulations implicitly assume that the underlying source signals essentially originate from spatially localized point sources albeit with their respective echoes and reflections. This assumption is usually not valid for strongly diffuse or spatially distributed noise sources like wind noise emanating from many directions at comparable sound pressure levels. For these types of distributed noise scenarios, the separation achievable with ICA approaches alone is insufficient.
What is desired is a simplified speech processing method that can separate speech signals from background noise in near real-time and that does not require substantial computing power, but still produces relatively accurate results and can adapt flexibly to different environments.